Alerting and mitigating divergence of anatomical feature locations from prior images to real-time interrogation

ABSTRACT

Systems, devices, methods, and computer program products for identifying and mitigating image-to-body divergence are disclosed herein. In some embodiments, a method includes receiving sensor data from a medical device while the medical device is inserted within an anatomic region of a patient and after it has been registered to an anatomic model of the anatomic region, where the anatomic model is based on previously-obtained image data of the anatomic region and includes a virtual path extending throughout the anatomic model to an anatomic structure of interest, and where sensor data indicates a location of at least a portion of the medical device; comparing the sensor data to a corresponding portion of the virtual path; based at least in part on the comparison, producing a divergence classifier indicative of a divergence of the anatomic region from the anatomic model; and generating an alert when the divergence classifier exceeds a predetermined threshold.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This patent document claims priority to and the benefit of U.S.Provisional Patent Application No. 63/065,420, filed Aug. 13, 2020, andincorporated herein by reference in its entirety.

TECHNICAL FIELD

The present disclosure is directed to systems, devices, methods, andcomputer program products for determining, alerting, predicting and/ormitigating divergence of an anatomical feature between a pastpre-operative image and a present physical location, particularly duringa minimally invasive medical procedure using a medical instrument.

BACKGROUND

Minimally invasive medical techniques are intended to reduce the amountof tissue that is damaged during medical procedures, thereby reducingpatient recovery time, discomfort, and harmful side effects. Suchminimally invasive techniques may be performed through natural orificesin a patient anatomy or through one or more surgical incisions. Throughthese natural orifices or incisions, an operator may insert minimallyinvasive medical tools to reach a target tissue location. Minimallyinvasive medical tools include instruments such as therapeutic,diagnostic, biopsy, and surgical instruments. Medical tools may beinserted into anatomic passageways and navigated toward a region ofinterest within a patient anatomy.

To assist with reaching the target tissue location, the location andmovement of the minimally invasive medical tools may be mapped withimage data of the patient anatomy, typically obtained prior to medicalprocedure. The image data may be used to assist navigation of themedical tools through natural or surgically-created passageways inanatomic systems such as the lungs, the colon, the intestines, thekidneys, the heart, the circulatory system, or the like. Yet, severalchallenges arise in reliably and accurately mapping the medical toolsand images of the anatomic passageways, particularly for locatinganatomic structures of interest identifiable in the previously-obtainedimage data.

SUMMARY

Disclosed herein are systems, devices, methods, and computer programproducts for identifying and mitigating divergence of anatomicstructures between prior pre-operative images (e.g., previously obtainedimages obtained using one or more medical imaging modalities, such ascomputed tomography (CT) scan images) of a patient's anatomy to sensordata, such as position, location, or shape sensor data obtained from oneor more sensors in a medical device while inside the patient's anatomyand/or intraoperative images obtained in real-time by a medical device.Also disclosed are systems, devices, methods, and computer programproducts for determining the divergence of anatomic structures from ananatomic model and predicting where their actual locations are while themedical device is interrogating the patient's anatomy in real-time.

In some embodiments, for example, a system for determining divergence ofan anatomic region from an anatomic model of the anatomic regionincludes a medical device comprising a sensor, wherein the medicaldevice is insertable within a patient's anatomy; and a computing devicein communication with the medical device, where the computing devicecomprises a processor and a memory, the memory coupled to the processorand storing instructions that, when executed by the processor, cause thesystem to perform operations comprising: receiving data acquired by thesensor of the medical device while the medical device is inserted withinan anatomic region of the patient and after the medical device has beenregistered to an anatomic model of the anatomic region, wherein theanatomic model is based on previously-obtained image data of theanatomic region and includes a virtual path extending throughout theanatomic model to an anatomic structure of interest, and wherein sensordata indicates a location of at least a portion of the medical device,comparing the sensor data to a corresponding portion of the virtualpath, based at least in part on the comparison, producing a divergenceclassifier indicative of a divergence of the anatomic region from theanatomic model, and generating an alert when the divergence classifierexceeds a predetermined threshold.

In some embodiments, for example, a non-transitory, computer-readablemedium can store instructions thereon that, when executed by one or moreprocessors of a computing system, cause the computing system to performoperations comprising: receiving data acquired by a sensor of a medicaldevice while the medical device is inserted within an anatomic region ofthe patient and after the medical device has been registered to ananatomic model of the anatomic region, wherein the anatomic model isbased on previously-obtained image data of the anatomic region andincludes a virtual path extending throughout the anatomic model to ananatomic structure of interest, and wherein sensor data indicates alocation of at least a portion of the medical device, comparing thesensor data to a corresponding portion of the virtual path, based atleast in part on the comparison, producing a divergence classifierindicative of a divergence of the anatomic region from the anatomicmodel, and generating an alert when the divergence classifier exceeds apredetermined threshold.

BRIEF DESCRIPTION OF THE DRAWINGS

Many aspects of the present disclosure can be better understood withreference to the following drawings. The components in the drawings arenot necessarily to scale. Instead, emphasis is placed on illustratingclearly the principles of the present disclosure. The drawings shouldnot be taken to limit the disclosure to the specific embodimentsdepicted, but are for explanation and understanding only.

FIG. 1 is a schematic representation of a portion of a medicalinstrument system inserted within an anatomic passageway of a patient.

FIG. 2 is a diagram illustrating a plurality of coordinate pointsforming a point cloud representing a shape of the portion of the medicalinstrument system of FIG. 1 configured in accordance with variousembodiments of the present technology.

FIG. 3 is a flow diagram illustrating a method for identifying and/ormitigating divergence of a target anatomic structure from apredetermined location along a planned path of the medical instrumentsystem of FIG. 1 in accordance with various embodiments of the presenttechnology.

FIGS. 4A-4C are diagrams showing virtual navigation images for a medicalinstrument system using an anatomic model that depicts divergence of atarget anatomic structure in accordance with various embodiments of thepresent technology.

FIGS. 5A and 5B are schematic representations of displays of a displaysystem in accordance with various embodiments of the present technology.

FIG. 6 is a schematic representation of a robotic or teleoperatedmedical system configured in accordance with various embodiments of thepresent technology.

FIG. 7 is a schematic representation of a manipulator assembly, amedical instrument system, and an imaging system configured inaccordance with various embodiments of the present technology.

DETAILED DESCRIPTION

The present disclosure is directed to systems, devices, methods, andcomputer program products for identifying and mitigating divergencebetween (i) an anatomic model generated from prior pre-operative imaging(e.g., obtained using one or more medical imaging modalities, such as CTscan imaging) of patient anatomy and (ii) sensor data, such as position,location, and shape sensor data and/or images obtained in real-time by amedical device positioned within the patient anatomy. Also disclosed aresystems, devices, methods, and computer program products for determiningthe divergence of anatomic structures and predicting where their actuallocations are while the medical device is interrogating the patient'sanatomy in real-time.

In some implementations, as illustrated by example embodiments below,the disclosed systems, devices, methods and computer program productscan be used to determine how target regions of pulmonary airways changefrom a pre-operative 3D map of the airways constructed before a medicalprocedure (from image data collected by an imaging system, e.g., CTsystem) to their actual locations and conformations during the medicalprocedure while the medical device, such as a robotic catheter, isnavigating in the airways based on the 3D map.

While the disclosed embodiments are described herein primarily based onidentifying and mitigating divergence of target anatomic structures inthe pulmonary airways for the purpose of facilitating understanding ofthe underlying concepts, it is understood that the disclosed embodimentscan also include identifying and mitigating divergence of targetanatomic structures in other tissues, organs and organ systems,including but not limited to the colon, the intestines, the kidneys, theheart, the urinary tract, and the circulatory system. Similarly, whilethe disclosed embodiments pertain to CT-to-body divergence, it isunderstood other imaging modalities are applicable to the disclosedtechniques, including but not limited to magnetic resonance imaging(MRI), X-Ray imaging, ultrasound imaging, and others.

Divergence is a phenomenon where a current location of an anatomicstructure within a patient has changed in relation to a previouslocation of the anatomic structure within the patient that was observedwithin previously-obtained (or pre-operative) images of the anatomicstructure. This phenomenon is referred to herein as “image-to-bodydivergence,” and in the case of specific imaging modalities like CTimaging, it can be referred to as “CT-to-body divergence.” In commonclinical practice, image-to-body divergence typically occurs whenmedical images are taken well before a medical procedure, such as weeksor months before the procedure. Image-to-body divergence may also occurshortly before a medical procedure for certain anatomic structures, suchas the lungs. For patients who are to undergo a medical proceduredirected to interrogating target anatomic structures (e.g., potentialtumor sites) identified within previously-obtained medical images (e.g.,CT images, MRI images, etc.), image-to-body divergence of the targetanatomic structure is a relatively common occurrence by the time thepatient undergoes the medical procedure. This can complicate or eventhwart the medical procedure, as target sites may be difficult orimpossible to locate by the physician performing the procedure usingcurrently-available medical device(s). For example, image-to-bodydivergence of a patient's lungs may occur on the same day as thepre-operative imaging, e.g., because of differences in the patient'smanner of breathing during pre-operative imaging (controlled breathingvia mechanical ventilation) and during the medical procedure (naturalbreathing), as well as due to effects of lung contraction andatelectasis—any of which can create and exacerbate image-to-bodydivergence, which in turn can greatly influence the medical procedure.

Image-to-body divergence can be a result of both inherent and externalcauses. For example, pulmonary structures naturally change shape in amedical procedure for a variety of reasons (referred to as “naturaldeformation”). These reasons include atelectasis, discrepancies betweenthe patient's body position and manner of breathing (e.g., breath hold,positive pressure ventilation versus spontaneous breathing) during theprior pre-operative imaging and during the medical procedure, and/orchanges in pulmonary anatomy itself, such as a change in the targettumor or lung since the prior imaging (especially when the imaging wasperformed several weeks to months prior to the medical procedure). Also,pulmonary structures can change shape due to “induced deformation.”Induced deformation can be caused by a medical device pushing on airwaywalls while performing the procedure and/or by an intubation angle thatcan change the tilt of airways in the patient's body.

For a pulmonary diagnostic procedure (e.g., for a biopsy of tissue in oraround airways of the lungs), pre-operative images of pulmonarystructures can be overlaid to create a 3D map or model of the pulmonarystructures that can be used to inform a medical device user (e.g., aphysician) where and how to navigate the medical device through thepulmonary structures to reach a target anatomic structure or region.Because the 3D map or model is based on pre-operative images,image-to-body divergence may cause the physician to drive the medicaldevice to a correct location within the 3D map or model where the systemhad mapped the target anatomic structure but at which the targetanatomic structure is no longer positioned, resulting in misseddiagnosis or improper intervention because the target anatomic structurewas not found or reached and/or an incorrect tissue sample was actedupon. Conventional techniques to create a 3D map and navigation planhave not addressed or have been unable to accurately account forimage-to-body divergence. Moreover, existing techniques to create a 3Dmap and navigation plan for a medical instrument do not provide themedical device user with adequate and timely notice of image-to-bodydivergence of the target structure being navigated towards.

The present technology provides techniques for identifying anddetermining the extent of image-to-body divergence, and for mitigatingsuch image-to-body divergence by generating alerts and/or predicting atrue location of target anatomical structures determined to havediverged from previously-obtained images, e.g., pre-operative images.For example, the disclosed techniques generate an anatomic model of ananatomic region based on pre-operative images of the anatomic region.Based on the pre-operative images, the disclosed techniques alsogenerate a planned path for a user to navigate a medical devicethroughout the anatomic region to arrive at a target anatomic structureidentified within the pre-operative images. The planned path isprojected onto the anatomic model. Once the medical device is positionedwithin the anatomic region and the medical device is registered to theanatomic model, a user can use the anatomic model and the planned pathto navigate the medical device toward the target anatomic structurewithin the anatomic region. At various points during the navigation(e.g., as the medical device passes anatomic landmarks, as the medicaldevice navigates a specified distance, as the medical device approachesthe end of the planned path, as the medical device approaches the targetanatomic structure, etc.), the disclosed techniques can compare (i) oneor more planned locations of one or more portions (e.g., a plannedlocation of the tip) of the medical device along the planned path to(ii) one or more current locations of the corresponding portion(s)(e.g., a current location of the tip) of the medical device extractedfrom position sensor data (e.g., shape data) captured by one or moresensors of the medical device. The difference(s) between the currentlocation(s) and the corresponding planned location(s) can provideindications of the magnitude and direction of divergence between theanatomic region and the anatomic model. In some implementations of thedisclosed techniques, the user is alerted when the determined divergenceis above a predetermined threshold, e.g., such as a predetermineddistance of divergence. Furthermore, because motion of the targetanatomic structure is correlated with motion of the anatomic regionlocal to the target anatomic structure, in some implementations, thedisclosed techniques can (i) evaluate the difference(s) between thecurrent location(s) and the planned location(s) as the medical deviceapproaches the planned position of the target anatomic structure topredict the target anatomic structure's current position and (ii) updatethe anatomic model to reflect the predicted position. Thus, the presenttechnology mitigates the effects of CT-to-body divergence on theanatomic model in a manner agnostic to the source or reason for thedivergence and facilitates navigation of the medical device to a likelycurrent position of the target anatomic structure, thereby increasingthe likelihood of a successful medical procedure.

These and other embodiments are discussed in greater detail by theexamples below.

A. EMBODIMENTS OF TECHNIQUES FOR IDENTIFYING AND MITIGATINGIMAGE-TO-BODY DIVERGENCE

FIG. 1 is a schematic representation of a portion of a medicalinstrument system 604 configured in accordance with various embodimentsof the present technology, which is inserted within an anatomic region150 (e.g., human lungs) of a patient, such as during a medicalprocedure. As shown in the diagram of FIG. 1 , the portion of themedical instrument system 604 inserted into the anatomical regionincludes an elongate device 131, which is extended within branchedanatomic passageways 152 of the anatomic region 150. In this example,the anatomic passageways 152 include a trachea 154 and a plurality ofbronchial tubes 156 of the lungs.

In some embodiments, the elongate device 131 is part of a flexiblecatheter or other biomedical device that can be sized and shaped toreceive a medical instrument and to facilitate delivery of the medicalinstrument to a distal portion 138 of the elongate device 131 forvarious purposes. For example, the medical instrument of the medicalinstrument system 604 can be used for medical procedures, such as forsurvey of anatomic passageways, surgery, biopsy, ablation, illumination,irrigation, and/or suction. The medical instrument can includepositional sensors, rate sensors, image capture probes, biopsyinstruments, laser ablation fibers, and/or other surgical, diagnostic,and/or therapeutic tools. Further details regarding the medicalinstrument system 604 are described in greater detail below inconnection with FIGS. 6 and 7 .

In some example embodiments (discussed in greater detail below inconnection with FIG. 7 ), the elongated device 131 can include anendoscope or other biomedical devices having one or more image capturedevices 747 positioned at the distal portion 138 of the elongated device131 (as in the example shown in FIG. 1 ) and/or at other locations alongthe elongated device 131. In these embodiments, the one or more imagecapture devices 747 can capture one or more real navigational images orvideo (e.g., a sequence of one or more real navigational image frames)of anatomic passageways and/or other real patient anatomy while theelongated device 131 is within the anatomic region 150 of the patient.

In the example implementation shown in FIG. 1 , the elongate device 131has a position, orientation, pose, and shape within the anatomic region150, all or a portion of which (in addition to or in lieu of movement,such as speed or velocity) can be captured as positional sensor data. Inthis or other examples, a positional sensor system 608 in communicationwith the medical instrument system 604 is configured to acquire theposition sensor data from at least one sensor of the elongate device131. In various embodiments of the medical instrument system 604, forexample, at least one sensor of the elongate device 131 can include ashape sensor 133 and/or one or more position measuring devices (eachdiscussed later in greater detail below in connection with FIG. 7 ). Insome implementations, the positional sensor system 608 can survey theanatomic passageways 152 by gathering positional sensor data of themedical instrument system 604 within the anatomic region 150 in a frameof reference of the medical instrument and/or elongated device 131,e.g., a cartesian coordinate system frame of reference (XM, YM, ZM). Thepositional sensor data may at least in part be recorded as a set oftwo-dimensional or three-dimensional coordinate points.

In the example of the anatomic region 150 being human lungs, thecoordinate points may represent the locations of the distal portion 138of the elongate device 131 and/or of other portions of the elongatedevice 131 while the elongate device 131 is advanced through the trachea154 and the bronchial tubes 156. In these and other embodiments, thecollection of coordinate points may represent the shape(s) of theelongate device 131 while the elongate device 131 is advanced throughthe anatomic region 150. Still, in these and other embodiments, thecoordinate points may represent positional data of other portions of themedical instrument system 604.

The set of 2D and/or 3D coordinate points from the recorded positionalsensor data may together be used to form a point cloud. For example,FIG. 2 is a diagram illustrating a plurality of coordinate points 262forming a point cloud 260 representing a shape of the portion of theelongate device 131 of FIG. 1 configured in accordance with variousembodiments of the present technology. In various implementations, forexample, the point cloud 260 is generated from the union of all or asubset of the coordinate points 262 recorded by the positional sensorsystem 608, e.g., while the elongate device 131 is in a stationaryposition.

In some embodiments, a point cloud (e.g., the point cloud 260) caninclude the union of all or a subset of coordinate points recorded bythe positional sensor system 608 during a data capture period that spansmultiple shapes, positions, orientations, and/or poses of the elongatedevice 131 within the anatomic region 150. In these embodiments, thepoint cloud can include coordinate points captured by the positionalsensor system 608 that represent multiple shapes of the elongate device131 while the elongate device 131 is advanced or moved through patientanatomy during the image capture period. Additionally, or alternatively,because the configuration, including shape and location, of the elongatedevice 131 within the patient may change during the image capture perioddue to anatomical motion, the point cloud in some embodiments cancomprise the plurality of coordinate points 262 captured by thepositional sensor system 608 that represent the shapes of the elongatedevice 131 as the elongate device 131 passively moves within thepatient.

A point cloud of coordinate points captured by the positional sensorsystem 608 can be registered to different models or datasets of patientanatomy. For example, a point cloud of coordinate points can beregistered to previously-obtained (pre-operative) image data of theanatomic region 150 captured by an imaging system. In someimplementations, for example, the previously-obtained image data of theanatomic region 150 is used to generate an anatomic model of theanatomic region. The elongate device 131 can be registered to theanatomic model based on the positional sensor data generated by thepositional sensor system 608 (and/or to endoscopic image data generatedby the one or more image capture devices 747, if applicable) to (i) mapthe tracked position, orientation, pose, shape, and/or movement of themedical instrument system 604 within the anatomic region 150 to acorrect position in real-time within the anatomic model, and/or (ii)determine a virtual navigational image of virtual patient anatomy of theanatomic region 150 from a viewpoint of the medical instrument system604 at a location within the anatomic model 150 corresponding to alocation of the elongate device 131 within the patient.

Referring back to FIG. 1 , the anatomic region 150 includes an anatomicstructure of interest 198 (also referred to as a “target anatomicstructure” or “target”), e.g., such as suspected tumorous tissue. Insome implementations, the target 198 is mapped to the anatomic model ofthe anatomic region 150 based on the physical location of the target 198relative to the anatomic region 150 observed in previously-obtained(pre-operative) image data. The anatomic model can further provide aplanned path for the user to navigate the elongated device 131 to thetarget anatomic structure 198 during a medical procedure. After themedical instrument system 604 is registered to the anatomic model, theplanned path can be updated based on the tracked position, orientation,pose, shape, and/or movement of the elongate device 131 within theanatomic region 150. The planned path can be manifested on virtualnavigational image(s) of virtual patient anatomy of the anatomic region150 from the viewpoint of the medical instrument system 604, which canbe represented as a line or series of points along one or more views ofthe anatomic region 150 of the generated anatomic model corresponding toa location of the medical instrument system 604 within the patient.Examples of virtual navigational image(s) associated with the anatomicmodel depicting a portion of an example planned path to the target 198are discussed in greater detail below in connection with FIGS. 4A-4Cand/or FIGS. 5A and 5B.

Yet, as discussed above, due to image-to-body divergence the user cannotbe certain that the target 198 is in the same location at the time ofthe medical procedure as it was at the time the previously-obtained,pre-operative image data was acquired. Thus, the user also cannot becertain that the target 198 is at the location depicted in the anatomicmodel and in the virtual navigational images. Therefore, the medicalinstrument system 604 can be implemented to identify and/or mitigatepotential image-to-body divergence based on the following techniques inaccordance with the present technology.

FIG. 3 is a flow diagram illustrating a method 300 for identifyingdivergence of an anatomic region from an anatomic model of the anatomicregion that was generated from previously-obtained, pre-operative imagedata of the anatomic region in accordance with various embodiments ofthe present technology. The flow diagram of FIG. 3 further illustrateshow the method 300 can be implemented to mitigate the identifieddivergence, e.g., such as alerting a user of a medical device and/orpredicting an actual position of the target anatomic structure, whichcan be used for redirecting navigation of a medical device, also inaccordance with various embodiments of the present technology.

The method 300 is illustrated as a set of operations or processes302-312, which can optionally include processes 314-322 in various waysor combinations. All or a subset of the processes of the method 300 canbe implemented by a medical instrument operating in conjunction with acomputing device, such as a control system in communication with orintegrated with a medical system comprising the medical instrument.Alternatively or in combination, all or a subset of the processes of themethod 300 can be implemented by a control system of a medicalinstrument system or device, including but not limited to variouscomponents or devices of a robotic or teleoperated system 600, asdescribed in greater detail below in connection with FIGS. 6 and 7 , aswell as any other suitable system. The computing device or controlsystem for implementing the method 300 can include one or moreprocessors operably coupled to a memory storing instructions that, whenexecuted, cause the computing system to perform operations in accordancewith some or all of the processes 302-310 and/or processes 312-322 ofthe method 300. Additionally, or alternatively, all or a subset of theprocesses of the method 300 can be executed by an operator (e.g., aphysician, a user, etc.) of the system 600. Furthermore, any one or moreof the processes of the method 300 can be executed in accordance withthe discussion above. Several of the processes 302-322 are discussedbelow with continual reference to one or more of FIGS. 4A-5B tofacilitate clarity and understanding of the present technology.

At process 302, the method 300 captures, receives, and/or processesimage data of an anatomic region of the patient from an imaging systemand generates an anatomic model. In some implementations, the imagingsystem is a CT imaging system or another imaging system. In someimplementations of the process 302, the image data can be captured,received, and/or processed during an image capture period of the imagingsystem. The image capture period can correspond to a time period duringwhich the imaging system is activated. In some embodiments, for example,the image capture period can be pre-operative such that the image datais captured, received, and/or processed before (e.g., minutes, hours,days, weeks, months, etc.) the medical procedure in which medicalinstrument system is advanced into the patient. In other embodiments,the image capture period can be intraoperative such that the image dataof the patient is captured, received, and/or processed while the medicalinstrument system is positioned within the patient. In theseembodiments, the medical instrument system may be stationary during theimage capture period, may be subject to commanded movement (e.g.,operator-commanded advancement or bending) during the image captureperiod, and/or may be passively moving (e.g., subject to no commandedmovement but subject to anatomical motion from respiratory activity,cardiac activity, or other voluntary or involuntary patient motion)during the image capture period. In still other embodiments, the imagecapture period can be postoperative such that the image data of thepatient is captured, received, and/or processed after the medicalinstrument system is removed from the patient. In some implementationsof the process 302, for example, the image data can be captured,received, and/or processed in real-time or near real-time.

The captured, received, and/or processed image data of the patient caninclude graphical elements representing anatomical features of thepatient and, in the case of intraoperative image data, the captured,received, and/or processed image data can include graphical elementsrepresenting the medical instrument system. In some implementations ofthe process 302, for example, the anatomic model of the anatomicalfeatures of the patient can be generated by segmenting and filtering thegraphical elements included in the image data. For example, during asegmentation process, pixels or voxels generated from the image data maybe partitioned into segments or elements and/or be tagged to indicatethat they share certain characteristics or computed properties such ascolor, density, intensity, and texture. In some embodiments, less thanall of the image data may be segmented and filtered. The segments orelements associated with anatomical features of the patient are thenconverted into an anatomic model, which is generated in an imagereference frame (XI, YI, ZI). Examples of the generated anatomic modelare depicted in FIGS. 5A and 5B, and are discussed in greater detailbelow.

At process 304, the method 300 identifies the position of one or moretarget anatomic structures relative to the anatomic region from theimage data and generates a planned path to the target(s) via anatomicpassageways of the generated anatomic model. The planned path provides amap for navigating a medical instrument (e.g., an elongated device 131of the medical instrument system 604 as shown in FIG. 1 ) throughout theimaged anatomic region, such as via anatomic passageways like pulmonaryairways, toward the target(s). In some implementations of the process304, for example, the planned path can be generated on a virtualnavigational image (or set of virtual navigational images) that providea virtual map of the patient's anatomy. FIG. 4A, discussed in greaterdetail below, illustrates an example of a planned path 452 overlaid on avirtual image of a portion of the anatomic region 150, e.g., lungs, ofthe patient.

At process 306, the method 300 records positional sensor data of amedical instrument system (e.g., the medical instrument system 604)positioned within the anatomic region and performs a registrationbetween the recorded positional sensor data and the image data. In someimplementations of the process 306, the positional sensor data isrecorded using a positional sensor system (e.g., the positional sensorsystem 608), which can be recorded during a position data capture periodof the positional sensor system. For example, the positional sensor dataprovides positional information (e.g., shape, position, orientation,pose, movement, etc.) of the medical instrument system while at least aportion of the medical instrument system is located within the anatomicregion. The position data capture period can correspond to a time periodduring which a shape sensor and/or one or more other positional sensorsof the positional sensor system are activated to collect and recordpositional sensor data. For example, during the position data captureperiod, the medical instrument system may be stationary, may be subjectto commanded movement (e.g., operator-commanded advancement or bending),and/or may be passively moving (e.g., subject to no commanded movementbut subject to anatomical motion from respiratory activity, cardiacactivity, or other voluntary or involuntary patient motion). In someimplementations of the process 306, the positional sensor data can be atleast partially recorded as one or more coordinate points in two orthree dimensions in the medical instrument reference frame (XM, YM, ZM),e.g., which can be related to a surgical reference frame (XS, YS, ZS) inexample implementations in a surgical environment. In these and otherimplementations, for example, a coordinate point corresponding to thepositional sensor data can be associated with a timestamp, which can beincluded as part of the recorded positional sensor data.

In some implementations of the process 306, the registration of themedical instrument system based on the recorded positional sensor dataand the image data involves aligning the medical instrument frame ofreference (XM, YM, ZM) (and/or the surgical reference frame (XS, YS,ZS)) with the image reference frame (XI, YI, ZI). In some embodiments ofthe process 306, the registration includes generating a point cloud ofthe recorded positional sensor data. In various implementations, forexample, the point cloud can be generated from the union of all or asubset of the coordinate points associated with the recorded positionalsensor data, e.g., during one or more position data capture periods ofthe positional sensor system. For example, the point cloud can representone or more shapes of the medical instrument system as the medicalinstrument system is stationary and/or is actively or passively movedwithin the patient. In these and other implementations, the point canrepresent the location of one or more portions (e.g., a tip) of themedical instrument system over time (e.g., over multiple data captureperiods). In various examples, the point cloud may be generated in twoor three dimensions in the medical instrument reference frame (XM, YM,ZM).

The medical instrument reference frame (XM, YM, ZM) can be registered tothe anatomic model in the image reference frame (XI, YI, ZI). Thisregistration may rotate, translate, or otherwise manipulate by rigidand/or non-rigid transforms coordinate points of the point cloud toalign the coordinate points with the anatomic model. The transforms maybe six degrees-of-freedom transforms, such that the point clouds may betranslated or rotated in any or all of X, Y, Z, pitch, roll, and yaw. Insome implementations of the registration between recorded positionalsensor data and the image data at the process 306, the method 300 usesan iterative closest point (ICP) algorithm to perform the registration.For example, the method 300 can (i) compute a point-to-pointcorrespondence between coordinate points in the point cloud to points(e.g., on a centerline or at other locations) within the anatomic modeland (ii) compute an optimal transform to minimize Euclidean distancesbetween corresponding points. The registration between the recordedpositional sensor data in the instrument frame of reference and theimage data in the image reference frame may be achieved, for example, byusing a point-based ICP technique, as described in U.S. Provisional Pat.App. Nos. 62/205,440 and 62/205,433, which are both incorporated byreference herein in their entireties. In other implementations of theprocess 306, the registration can be performed using another technique.

At process 308, the method 300 captures positional sensor data atvarious times as the medical instrument system (e.g., the elongateddevice 131 of the medical instrument system 604) is navigated (e.g.,driven) along the planned path generated at the process 304 en route tothe target(s). In some implementations, these various times correspondto times the medical instrument system is positioned at anatomiclandmarks, has navigated a specified distance, is approaching the end ofthe planned path, is approaching the target(s), and/or to otherspecified events. For example, in various implementations of the process308, the medical instrument system is navigated along the planned pathgenerated at the process 304 en route to a target when it is determinedthat the medical instrument system is proximate a recognizableanatomical landmark. In some examples pertaining to the anatomic regionbeing anatomic passageways, the specified landmark may include a branchdivision point in the anatomic passageways. In examples where theanatomic passageways are pulmonary airways of the lungs, as in theanatomic region 150 of FIG. 1 , the specified landmark may include acarina. For example, in such implementations, the process 308 caninclude a verification technique where the operator of the medicalinstrument system checks to ensure the medical instrument system isbeing driven along the correct pathway in accordance with the plannedpath (en route to the target(s)). In some embodiments, the verificationtechnique can include instructing the operator to drive the medicalinstrument system to nearest landmarks along the navigated path, such ascarinas in the driven path, and obtain image(s) using the one or moreimage capture devices 747 to compare with expected carinas along theplanned path based on the virtual navigational image(s) associated withthe virtual map of the patient's anatomy.

FIGS. 4A-4C are provided to help further illustrate certain aspects ofthe method 300 of FIG. 3 . FIG. 4A, for example, is a partiallyschematic diagram showing an example of a virtual navigation image for amedical instrument system based on an anatomic model 450 (e.g.,generated at the process 302 and/or the process 304 of the method 300)of the anatomic region. The virtual navigation image includes a plannedpath 452 that navigates anatomic passageways 152 of the anatomic model450 towards a virtual target location 455. The virtual target location455 is positioned at a location relative to the anatomic model 450 at alocation that corresponds to a location of a target anatomic structure198 relative the anatomic region that was previously determined frompre-operative image data of the anatomic region. Also shown in FIG. 4Ais point cloud 260 composed of the plurality of positional coordinatepoints 262 that are associated with a current or recent position (e.g.,shape) of the medical instrument system within the anatomic region. Oneof more of the coordinate points 262 can be captured as the medicalinstrument system is navigated through the anatomic region in accordancewith the planned path 452. For example, the coordinate points 262 of thepoint cloud 260 can be captured as the medical instrument systemapproaches or reaches the end of the planned path 452.

Continuing with the above example, although the medical instrumentsystem has been navigated to a location within the anatomic region thatcorresponds to the end of the planned path 452 within the anatomic model450, the position of the point cloud 260 in the image frame of referencedivergences from the position of the planned path 452 in the image frameof reference. That is, the anatomic region has diverged from theanatomic model 450 at least along the portion of the anatomic regionthat corresponds to the illustrated point cloud 260. Also, because theposition of the target 198 (FIG. 1 ) moves in correlation with theanatomic region local to the target, the real position of the target(represented by real target location 435 in FIG. 4A) has also divergedfrom the virtual target location 455. Thus, should a user attempt tonavigate to the virtual target location 455 from the current location ofthe medical instrument system shown by the point cloud 260, the medicalinstrument system is unlikely to encounter the target that is nowpositioned at a location corresponding to the real target location 435shown in the virtual image. As such, the chances of a successful medicalprocedure (e.g., biopsy of the target 198 (FIG. 1 ), ablation of thetarget 198, etc.) in this scenario are significantly reduced.

That said, the difference between the planned path 452 and the pointcloud 260 can provide an indication of the direction and magnitude ofthe divergence between the anatomic region and the anatomic model.Therefore, referring to FIGS. 3 and 4A together, the method 300 atprocess 310 compares the captured positional sensor data (e.g., all or asubset of the coordinate points 262 of point cloud 260) to the plannedpath 452. In implementations of the process 310, the comparison betweenthe positional sensor data and the planned path can match positionalsensor data to one or more points along the planned path 452. Forexample, the comparison can match one or more points along the plannedpath that correspond to specified landmarks (e.g., carina(s)) topositional sensor data (e.g., one or more coordinate points 262) of themedical instrument system nearest to the specified landmarks. In someexamples, the positional sensor data (e.g., the one or more coordinatepoints 262) of the medical instrument system used in the comparison withthe specified landmarks correspond to position(s) of the distal portion138 of the elongate device 131 (FIG. 1 ). Additionally or alternatively,the comparison between the positional sensor data and the planned path452 can match an end point of the planned path 452 to positional sensordata (e.g., a coordinate point 262) corresponding to the location of thedistal portion 138 (e.g., a distal end or another portion) of themedical instrument system. In this manner, the process 310 can beimplemented using implementations of the medical instrument system thatmay only include a single sensor—such as a positional sensor or ratesensor located at a known location relative to the dimensions of themedical instrument system—as well as using embodiments of the medicalinstrument system that include multiple sensors and/or shape sensors.For example, the process 310 can be implemented using one or morepositions determined by a single sensor such as an electromagnetic (EM)sensor, e.g., preferably at the distal portion 138 of the elongatedevice 131, to determine an offset with the planned path using adivergence vector from that position of the elongate device. Yet, inanother example, the process 310 can be implemented using one or morepositional data points along the body of the elongate device 131acquired by the shape sensor 133, which can be correlated to the shapeof the airway by comparing to a greater portion of the planned path.Additionally or alternatively, for example, the comparison between thepositional sensor data and the planned path 452 can include sampling asubset of the positional sensor data and/or of the points along theplanned path 452 such that a first number of positional sensor datapoints are matched to the same number of points along the planned path452.

In some implementations, the comparison of the planned path 452 to thepositional sensor data (e.g., to coordinate points 262 of the pointcloud 260) can include determining an offset from the planned path tothe position of the medical instrument system indicated by thepositional sensor data. In one embodiment, for example, the offset canbe determined using vectors (referred to hereinafter as “divergencevectors”) pointing from one or more points along the planned path to thedetermined position of one or more portions of the medical instrumentsystem indicated by one or more corresponding positional sensor datapoints (e.g., coordinate points 262), or vice versa. For example, thedivergence vectors can be represented as:

Divergence Vector x,y,z=Positional Sensor Data Point x,y,z−Matched Pointx,y,z along Planned Path, where there is a Divergence Vector x,y,z forevery pair of matched points.

FIG. 4B is a schematic diagram showing the virtual navigation image ofFIG. 4A. As shown in FIG. 4B, coordinate points 262 of the point cloud260 are matched with corresponding points along the planned path 452.For example, coordinate points 262 of the point cloud 260 are matchedwith the nearest points along the planned path 452. In these and otherimplementations, the coordinate point 262 at the end of the point cloud260 is matched with a point at the end of the planned path 452. In theseand still other implementations, one or more points along the plannedpath 452 corresponding to one or more anatomic landmarks within theanatomic model 450 are matched with the nearest coordinate points 262 ofthe point cloud. As discussed above, should there be more points alongthe planned path 452 than coordinate points 262 in the point cloud 260(or vice versa), the points along the planned path 452 or the coordinatepoints 262 can be down-sampled such that there a number of points alongthe planned path 452 are matched with the same number of coordinatepoints 262 in the point cloud 260.

Divergence vectors 473 are shown in FIG. 4B between points along theplanned path 452 and corresponding coordinate points 262 in the pointcloud 260. Each divergence vector 473 can be calculated using theformula provided above. Furthermore, each divergence vector 473 providesan indication (magnitude and direction) of the divergence of theanatomic region from the anatomic model 450 at a location correspondingto the coordinate points 262 in the point cloud 260.

As discussed in greater detail below, the relevance of a givendivergence vector 473 can differ from the relevance of other divergencevectors 473 illustrated in FIG. 4B. For example, divergence vectors 473corresponding to coordinate points 262 in the point cloud 260 thatindicate the position of the distal portion (e.g., a distal end) of themedical instrument system can have a higher relevance than divergencevectors 473 that correspond to coordinate points 262 in the point cloud260 that indicate the position of more proximal portions of the medicalinstrument system. This is because anatomic passageways 152 aregenerally expected to decrease distally in size (e.g., diameter), whichis expected to make distal portions of the anatomic passageways 152 moresusceptible to CT-to-body divergence. Another reason the relevance oftwo divergence vectors 473 can differ is that movement of a portion ofthe anatomic region local to a target (e.g., target 435) is expected toimpact the position of the target to a greater extent than movement ofanother portion of the anatomic region further away from the target.Thus, divergence vectors 473 corresponding to portions of the medicalinstrument system proximate the target (e.g., target 435) are expectedto provide a better indication of the current position of the targetthan divergence vectors 473 corresponding to portions of the medicalinstrument system further from the target. Thus, the method 300 canconsider all or a subset of the divergence vectors 473 generated by theprocess 310 of the method 300 (e.g., a set number of divergence vectors473 nearest the target 198, another anatomic landmark, or the distalregion (e.g., the distal end) of the medical instrument system; alldivergence vectors within a specified distance of the target (e.g.,target 435), another anatomic landmark, or the distal region (e.g., thedistal end) of the medical instrument system; etc.), and/or the method300 can apply different weightings to the divergence vectors 473.

In this manner, for example, the method 300 can detectclinically-relevant divergence indicative of significant tissuedeformation in the probed anatomic passageways and, likely, a change inposition of the target anatomic structure from its determined locationfrom the previously-obtained image data. Referring again to FIG. 3 , atprocess 312, the method 300 identifies whether there is divergence. Themethod 300 may end at process 314 when no divergence is detected, orcontinue to process 316 when divergence is detected. It is understoodthat the method 300 can be repeated intermittently or continuously, suchthat identification of divergence is performed (at process 312) for aplurality of landmarks (e.g., for all or a subset of the carinasencountered by the medical instrument system as it is driven along theplanned path), after the medical instrument system has traversed aspecified distance; as the medical instrument system reaches the end ofthe planned path 452, as the medical instrument system approaches thetarget, etc.

To detect significant divergence, for example, some embodiments of themethod 300 include a divergence classification technique that isimplemented at process 312. For example, the divergence classificationtechnique can classify (e.g., determine, predict, etc.) (i) themagnitude and/or direction of the divergence of a portion (e.g.,anatomic passageways) of the anatomic region from a correspondingportion of the anatomic model and/or (ii) the magnitude and/or directionof the divergence of the actual location (e.g., real target location435; FIGS. 4A and 4B) of a target from its virtual location (e.g.,virtual target location 455; FIGS. 4A and 4B) within or relative to theanatomic model. The divergence classification technique may includefinding one or more matching points between the planned path (e.g.,virtual line of the airway) and the positional sensor data produced bythe position sensor(s) of the medical instrument system, as discussedabove. After determining the matching points, the technique may includecalculating one or more divergence vectors, which can include vector(s)between one or more points along the planned path and correspondingpositional sensor data point(s) (e.g., indicative of the position and/orshape of the elongated device 131), also discussed above. Notably, thetechnique may implement other ways to identify divergence of thepositional sensor data from the pre-operative image data and/or otherways to update the location of the target, such as using simpletranslation of the end of the planned path onto one or more coordinatepoints corresponding to the distal region 138 of the elongated device131 (FIG. 1 ). In the example divergence vector implementations, thedetermined divergence vectors may be different than those that usepoint-by-point vectorization of the planned path to the positionalsensor data. For example, linear or quadratic interpolations can be usedto determine the differences between the positional sensor data and theplanned path. The divergence classification technique produces aquantitative value, referred to as a “divergence classifier,” which canbe a scalar or vector representation of the divergence of the anatomicregion and/or of the target from the planned path (from their determinedlocation in the previously-obtained image data) based on the divergencevectors calculated in accordance with some example embodiments of theprocess 310.

In some embodiments of the divergence classification technique,implemented at the process 312, the method 300 determines an offsetbetween the portion of the virtual path and the position sensor data ofthe medical device, where the offset is a quantitative value that maycorrespond to an amount or degree of divergence between anatomicfeatures determined from the previously-obtained image data and the sameanatomic features probed by the insertable medical instrument whilebeing driven along the planned path. In some implementations, thedivergence classifier includes a device-to-path distance parameter,which includes a mean distance between at least one point on the plannedpath to the one or more position data points of the medical instrumentsystem (e.g., at the location of the distal portion 138 of the elongatedevice 131), which can be determined using one or more divergencevectors calculated at process 310. The device-to-path distance parametercan be represented as x cm or other distance unit. In some embodiments,for example, the device-to-path distance parameter includes a range ofmean distance values. Yet, in some implementations, the device-to-pathdistance parameter can include a mean distance among weighted divergencevectors. For example, as discussed above, divergence vectors closer tothe target (e.g., at the distal end of the elongate device) may beweighted higher than divergence vectors further from the target.

Because the divergence classifier is a quantitative value, thedivergence classifier can be applied to identify whether there issignificant divergence of the target or other portion of the anatomicregion probed by the medical instrument system. For example, if thedevice-to-path distance parameter is determined to be relatively smallbased on a divergence threshold, e.g., less than 1 mm, then the method300 can determine there is no significant divergence at process 312 andterminate the method 300 (e.g., at least for this juncture of themedical procedure) at the process 314. Notably, the divergence thresholdcan vary for different anatomic regions or for different patients. Insome examples for pulmonary airways, the divergence threshold todetermine significant divergence of a target can be 1 cm or greater.Similarly, the divergence threshold can be a set of ranges, rather thana specific value, to inform of a degree of divergence. For example, thedivergence threshold can include 0 mm to <5 mm in one range representingnegligible divergence, 5 mm to <10 mm in a next range representinginsignificant divergence, 10 mm to <20 mm in another range representingsignificant divergence, and so forth.

Also, for example, the divergence classifier can be used to determine ifthe user should be alerted to the divergence based on the amount ordegree of the divergence. Moreover, for example, this feature may beused to derive metrics of divergence based on other clinical factors, ormay be used as a signal to switch between different registrationtechniques for the medical device registration protocol.

In some embodiments, the method 300 may optionally include process 316.At process 316, the method 300 triggers a divergence alert whendivergence is detected at 312. In some implementations of the process316, the method 300 generates an alert when the divergence classifierexceeds a predetermined threshold. For example, in some implementations,the method 300 compares a predetermined divergence threshold of 1 cm orgreater to the divergence classifier to trigger an alert to the user ofthe medical instrument system that the target anatomic structure 198 hassignificantly diverged from the planned path. FIGS. 5A and 5B, discussedin greater detail below, illustrate examples of alerting the user inexample implementations of the process 316.

The method 300 may optionally include processes 318-322 directed toupdating the anatomic model based on the comparison performed at theprocess 310 and/or on the divergence detected at the process 312. Asshown in the flow diagram of FIG. 3 , the method 300 identifies whetherto update the anatomic model (e.g., a virtual image of the anatomicmodel and/or the target) at process 318. If the method 300 determinesthat updating the anatomic models is not warranted, the method 300 mayend at block 320. Otherwise, the method 300 can proceed to implementprocess 322 to update the anatomic model by updating the depiction of acorresponding portion of the anatomic region, updating the planned path,and/or updating the virtual target position to a predicted position ofthe target (e.g., shown as predicted target location 445 in FIG. 4C,which is discussed in greater detail below). In some implementations,the process 318 may determine that updating the anatomic model iswarranted based on the divergence classifier (e.g., when the identifieddivergence exceeds a specified threshold) or based on user input (e.g.,provided via a display of the system, as discussed in greater detailbelow in connection with FIG. 5A). It is understood that the method 300can be repeated intermittently or continuously, such that thedetermination whether to update the anatomic model is performed (atprocess 318) continuously or intermittently.

At process 322, the method 300 updates the anatomic model. In someimplementations, this can include updating a corresponding portion ofthe anatomic region depicted in the anatomic model, updating the plannedpath projected onto the anatomic model, and/or updating the virtualtarget position 455 (FIGS. 4A-4C) to a predicted target location 445(FIG. 4C) that is expected to better align with the target's actual orreal target location 435 (FIGS. 4A-4C). For example, the method 300 mayimplement a technique to determine a predicted location of a targetanatomic structure (e.g., a predicted real location), where the targethas moved due to image-to-body divergence.

In some implementations, for example, the prediction technique findsmatching points between the planned path (e.g., a virtual line orcenterline of the anatomic passageway) and the position sensor data(e.g., shape data) produced by the sensor of the medical instrumentsystem, as discussed in greater detail above. After determining thematching points, the technique calculates divergence vectors, which caninclude vector(s) between point(s) from the planned path to the one ormore position data points associated with the shape of the medicalinstrument system. Notably, the technique may implement other ways tocompare differences in the shape data to the previously-obtained imagedata and update the virtual target, such as using simple translation ofthe end of the planned path to the target onto the last piece of thecatheter shape. In the divergence vector embodiments, the determineddivergence vectors may be different than those that use point-by-pointvectorization of the planned path to the medical instrument's shape. Forexample, linear or quadratic interpolations can be used to determine thedifferences in the position sensor data from the virtual planned path ofthe airways. The technique can update the virtual target position inreal time as the user (e.g., physician) moves closer to the targettissue. Also, the virtual target position can be updated (e.g., bypushing a button or selection of a software tool) once the surgeon isclose enough to the virtual target.

For example, in some embodiments of the method 300, the method furtherincludes updating the virtual location of the target anatomic structure,which can include selecting data including a last one or more (n)vectors associated with points nearest to the target, fitting a curve tothe selected data (wherein the curve includes a linear and quadratic),and extrapolating the curve to estimate a new target position of theanatomic structure in x,y,z coordinate points. Furthermore, in these andother embodiments, the updating the virtual location further includesapplying a weighting value to one or more of the vectors.

FIG. 4C is a schematic diagram of the virtual navigation images of FIGS.4A and 4B. As shown, the anatomic model 450 has been updated in FIG. 4Cto align with the positional sensor data (e.g., the point cloud 260). Inother words, the anatomic model 450 has been updated to illustratedeformation of anatomic passageways 152 local to the distal region ofthe medical instrument system. The planned path 452 has also beenupdated such that it better aligns with the positional sensor data.Furthermore, FIG. 4C illustrates a predicted target location 445 thatrepresents an updated position of the virtual target location 455 of thetarget along a target divergence vector 463. The target divergencevector 463 is calculated using the plurality of divergence vectors 473illustrated in FIG. 4B. For example, in some implementations, the targetdivergence vector 463 is calculated based on an average of all or subsetof the divergence vectors 473 from FIG. 4B. For example, all or a subsetof the divergence vectors 473 may be weighted in order to determine thetarget divergence vector 463. In some implementations, the weightings ofdivergence vectors 473 may be based on a distance between the end or tipof the elongate device 131 from the point of the divergence vectorspanning from the elongate device 131, e.g., where divergence vectorscloser to the end or tip are weighted greater. In these and otherimplementations, the weightings of the divergence vectors 473 may bebased on a distance between the virtual target location 455 and a pointof a corresponding divergence vector 473, e.g., where the divergencevectors closer to the virtual target location 455 are weighted heavier.Such distance-based weightings can be assigned in groups, e.g., based ondistance, or individually weighted.

FIG. 5A is a schematic representation of an example display 510illustrating a graphic user interface (GUI) having a GUI featureindicative of a divergence alert in accordance with various embodimentsof the present technology. The display 510 can be produced by a displaysystem (e.g., display system 610, discussed in greater detail inconnection with FIG. 6 ) in communication with the medical instrumentsystem (e.g., medical instrument system 604). As shown in FIG. 5A, thedisplay 510 includes a real navigational image 570, a composite virtualnavigational image 591 (also referred to as a “composite virtual image591”), a virtual navigational image 592, and a divergence alert GUIfeature 585.

The composite virtual image 591 of FIGS. 5A and 5B is displayed in theimage reference frame (XI, YI, ZI) and includes an anatomic model 550generated from image data of the anatomic region 150 (FIG. 1 ), e.g.,captured by an imaging system. The anatomic model 550 is registered(i.e., dynamically referenced) with a point cloud of coordinate points(e.g., the point cloud 260 of FIG. 2 ) generated by the positionalsensor system 608 (FIG. 1 ) to display a representation 504 within theanatomic model 550 of the tracked position, shape, pose, orientation,and/or movement of the medical instrument system 604 (e.g., of theelongate device 131) within the patient (FIG. 1 ). In FIG. 5A, theanatomic model 550 of the imaged anatomic region 150 of the patientincludes the target 198 as determined from the pre-operative imagesshown with respect to virtual anatomic passageways 552 spanning frombronchial tubes 557 to deep passageways 558. In FIG. 5B, the anatomicmodel 550 of the imaged anatomic region 150 of the patient includes anupdated target 598 as determined from implementation of the process 322,also shown with respect to virtual anatomic passageways 552 spanningfrom bronchial tubes 557 to deep passageways 558.

The divergence alert GUI feature 585 is shown in FIG. 5A on the display510 proximate the composite virtual image 591, which for example, can beimplemented as an icon, dialog box, or other manifestations thatindicate the identification of divergence in accordance with processes312-316 described above with reference to FIG. 3 . In the particularexample shown in FIG. 5A, the divergence alert GUI feature 585(optionally) includes an updated registration prompt, allowing a user toindicate whether they want the system to update the registration of themedical instrument system in accordance with the process 318-322. Theuser can select (a) interactive GUI feature 587 a to update theregistration (e.g., of the target anatomic feature 198 and/or theplanned path in the display 510) or (b) interactive GUI feature 587 b tonot update the registration.

FIG. 5B is a schematic representation of the example display 510 of FIG.5A, but illustrating the display after the user selects interactive GUIfeature 587 a, which is indicative of an affirmative command to updatethe registration of anatomic model to the position sensor data, thelocation of the target 198, and/or the planned path in the display 510(e.g., at least in the composite virtual navigational image 591).

Referring to both FIGS. 5A and 5B together, the real navigational image570 illustrates real patient anatomy (e.g., a carina 571 marking abranching point of two anatomic passageways 152) from a viewpointoriented distally away from the distal portion 138 of the elongatedevice 131 (FIG. 1 ). For example, the real navigational image 570 canbe captured by the one or more image capture devices 747 (e.g.,embodiments of endoscopic imaging system 609, discussed in greaterdetail in connection with FIG. 6 ) and provided to the display system610 to be presented on the display 510 in real-time or near real-time.

In some embodiments, the composite virtual image 591 is generated by avirtual visualization system (e.g., virtual visualization system 615,discussed in greater detail in connection with FIG. 6 ) of the controlsystem 612 (FIG. 6 ). Generating the composite virtual image 591 caninvolve registering the image reference frame (XI, YI, ZI) with thesurgical reference frame (XS, YS, ZS) and/or to the medical instrumentreference frame (XM, YM, ZM). For example, this registration may rotate,translate, or otherwise manipulate by rigid and/or non-rigid transformscoordinate points of the point cloud (e.g., the coordinate points 262 ofthe point cloud 260 of FIG. 2 ) captured by the positional sensor system608 to align the coordinate points with the anatomic model 550. Theregistration between the image and surgical/instrument frames ofreference may be achieved, for example, by using an ICP technique, oranother point cloud registration technique.

As further shown in FIGS. 5A and 5B, the virtual navigational image 592illustrates virtual patient anatomy, such as a virtual carina 501(corresponding to the real carina 571) marking a branching point of twovirtual anatomic passageways 552 (corresponding to real anatomicpassageways 152) of the anatomic model 550, from substantially the samelocation at which the real navigational image 570 is captured by theimage capture device 747 (FIG. 1 ). Thus, the virtual navigational image592 provides a rendered estimation of patient anatomy visible to theimage capture device 747 at a given location within the anatomic region150. Because the virtual navigational image 592 is based, at least inpart, on the registration of a point cloud generated by the positionalsensor system 608 and image data captured by the imaging system 618(FIG. 1 ), the correspondence between the virtual navigational image 592and the real navigational image 570 provides insight regarding theaccuracy of the registration.

As further shown in FIGS. 5A and 5B, the virtual navigational image 592can optionally include a navigation path overlay 599 (e.g., a plannedpath). In some embodiments, the navigation path overlay 599 is used toaid an operator to navigate the medical instrument system 604 throughanatomic passageways of an anatomic region 150 to the target anatomicstructure 198 within the patient. For example, the navigation pathoverlay 599 can illustrate a “best” path through an anatomic region foran operator to follow to deliver the distal portion of the elongateddevice 132 to a target location within the patient. In some embodiments,the navigation path overlay 599 can be aligned with a centerline of oranother line along (e.g., the floor of) a corresponding anatomicpassageway.

B. EMBODIMENTS OF ROBOTIC OR TELEOPERATED MEDICAL SYSTEMS FORIMPLEMENTING IMAGE-TO-BODY DIVERGENCE IDENTIFICATION AND MITIGATIONTECHNIQUES DURING A MEDICAL PROCEDURE

FIG. 6 is a schematic representation of a robotic or teleoperatedmedical system 600 (“medical system 600”) configured in accordance withvarious embodiments of the present technology. As shown, the medicalsystem 600 includes a manipulator assembly 602, the medical instrumentsystem 604 (from FIG. 1 ), a master assembly 606, and a control system612. The manipulator assembly 602 supports the medical instrument system604 and drives the medical instrument system 604 at the direction of themaster assembly 606 and/or the control system 612 to perform variousmedical procedures on a patient 603 positioned on a table 607 in asurgical environment 601. In this regard, the master assembly 606generally includes one or more control devices that can be operated byan operator 605 (e.g., a physician) to control the manipulator assembly602. Additionally, or alternatively, the control system 612 includes acomputer processor 614 and at least one memory 616 for effecting controlbetween the medical instrument system 604, the master assembly 606,and/or other components of the medical system 600. The control system612 can also include programmed instructions (e.g., a non-transitorycomputer-readable medium storing the instructions) to implement any oneor more of the methods described herein, including instructions forproviding information to a display system 610 and/or processing data forregistration of the medical instrument system 604 with the anatomicalmodel of the patient 603 (as previously described above). Themanipulator assembly 602 can be a teleoperated, a non-teleoperated, or ahybrid teleoperated and non-teleoperated assembly. Thus, all or aportion of the master assembly 606 and/or all or a portion of thecontrol system 612 can be positioned inside or outside of the surgicalenvironment 601.

To aid the operator 605 in controlling the manipulator assembly 602and/or the medical instrument system 604 during an image-guided medicalprocedure, the medical system 600 may further include a positionalsensor system 608, an endoscopic imaging system 609, an imaging system618, and/or a virtual visualization system 615. In some embodiments, thepositional sensor system 608 includes a location sensor system (e.g., anelectromagnetic (EM) sensor system) and/or a shape sensor system forcapturing positional sensor data (e.g., position, orientation, speed,velocity, pose, shape, etc.) of the medical instrument system 604. Inthese and other embodiments, the endoscopic imaging system 609 includesone or more image capture devices (not shown) that record endoscopicimage data that includes concurrent or real-time images (e.g., video,still images, etc.) of patient anatomy. Images captured by theendoscopic imaging system 609 may be, for example, two orthree-dimensional images of patient anatomy captured by an image capturedevice positioned within the patient 603, and are referred to as “realnavigational images,” such as the real navigation images 570 shown inFIGS. 5A and 5B.

In some embodiments, the medical instrument system 604 may includecomponents of the positional sensor system 608 and/or components of theendoscopic imaging system 609. For example, components of the positionalsensor system 608 and/or components of the endoscopic imaging system 609can be integrally or removably coupled to the medical instrument system604. Additionally, or alternatively, the endoscopic imaging system 609can include a separate endoscope (not shown) attached to a separatemanipulator assembly (not shown) that can be used in conjunction withthe medical instrument system 604 to image patient anatomy. Thepositional sensor system 608 and/or the endoscopic imaging system 609may be implemented as hardware, firmware, software, or a combinationthereof that interact with or are otherwise executed by one or morecomputer processors, such as the computer processor(s) 614 of thecontrol system 612.

The imaging system 618 of the medical system 600 may be arranged in thesurgical environment 601 near the patient 603 to obtain real-time and/ornear real-time images of the patient 603 before, during, and/or after amedical procedure. In some embodiments, the imaging system 618 includesa mobile C-arm cone-beam computerized tomography (CT) imaging system forgenerating three-dimensional images. For example, the imaging system 618can include a DynaCT imaging system from Siemens Corporation, or anothersuitable imaging system. In these and other embodiments, the imagingsystem 618 can include other imaging technologies, including magneticresonance imaging (MM), fluoroscopy, thermography, ultrasound, opticalcoherence tomography (OCT), thermal imaging, impedance imaging, laserimaging, nanotube X-ray imaging, and/or the like.

The virtual visualization system 615 of the control system 612 providesnavigation and/or anatomy-interaction assistance to the operator 605when controlling the medical instrument system 604 during animage-guided medical procedure. As described in greater detail below,virtual navigation using the virtual visualization system 615 can bebased, at least in part, upon reference to an acquired pre-operative orintra-operative dataset (e.g., based, at least in part, upon referenceto data generated by the positional sensor system 608, the endoscopicimaging system 609, and/or the imaging system 618) of anatomicpassageways of the patient 603. In some implementations, for example,the virtual visualization system 615 processes pre-operative and/orintraoperative image data of an anatomic region of the patient 603captured by the imaging system 618 to generate an anatomic model (notshown) of the anatomic region. The virtual visualization system 615 thenregisters the anatomic model to positional sensor data generated by thepositional sensor system 608 and/or to endoscopic image data generatedby the endoscopic imaging system 609 to (i) map the tracked position,orientation, pose, shape, and/or movement of the medical instrumentsystem 604 within the anatomic region to a correct position within theanatomic model, and/or (ii) determine a virtual navigational image ofvirtual patient anatomy of the anatomic region from a viewpoint of themedical instrument system 604 at a location within the anatomic modelcorresponding to a location of the medical instrument system 604 withinthe patient 603.

The display system 610 can display various images or representations ofpatient anatomy and/or of the medical instrument system 604 that aregenerated by the positional sensor system 608, by the endoscopic imagingsystem 609, by the imaging system 618, and/or by the virtualvisualization system 615. In some embodiments, the display system 610and/or the master assembly 606 may be oriented so the operator 605 cancontrol the manipulator assembly 602, the medical instrument system 604,the master assembly 606, and/or the control system 612 with theperception of telepresence.

As discussed above, the manipulator assembly 602 drives the medicalinstrument system 604 at the direction of the master assembly 606 and/orthe control system 612. In this regard, the manipulator assembly 602 caninclude select degrees of freedom of motion that may be motorized and/orteleoperated and select degrees of freedom of motion that may benon-motorized and/or non-teleoperated. For example, the manipulatorassembly 602 can include a plurality of actuators or motors (not shown)that drive inputs on the medical instrument system 604 in response tocommands received from the control system 612. The actuators can includedrive systems (not shown) that, when coupled to the medical instrumentsystem 604, can advance the medical instrument system 604 into anaturally or surgically created anatomic orifice. Other drive systemsmay move a distal portion (not shown) of the medical instrument system604 in multiple degrees of freedom, which may include three degrees oflinear motion (e.g., linear motion along the X, Y, Z Cartesian axes) andthree degrees of rotational motion (e.g., rotation about the X, Y, ZCartesian axes). Additionally, or alternatively, the actuators can beused to actuate an articulable end effector of the medical instrumentsystem 604 (e.g., for grasping tissue in the jaws of a biopsy deviceand/or the like).

FIG. 7 is a schematic representation of the manipulator assembly 602,the medical instrument system 604, and the imaging system 618 of FIG. 6within the surgical environment 601 and configured in accordance withvarious embodiments of the present technology. As shown in FIG. 7 , thesurgical environment 601 has a surgical frame of reference (XS, YS, ZS)in which the patient 603 is positioned on the table 607, and the medicalinstrument system 604 has a medical instrument frame of reference (XM,YM, ZM) within the surgical environment 601. During the medicalprocedure, the patient 603 may be stationary within the surgicalenvironment 601 in the sense that gross patient movement can be limitedby sedation, restraint, and/or other means. In these and otherembodiments, cyclic anatomic motion of the patient 603, includingrespiration and cardiac motion, may continue unless the patient 603 isasked to hold his or her breath to temporarily suspend respiratorymotion.

The manipulator assembly 602 includes an instrument carriage 726 mountedto an insertion stage 728. In the illustrated embodiment, the insertionstage 728 is linear, while in other embodiments, the insertion stage 728is curved or has a combination of curved and linear sections. In someembodiments, the insertion stage 728 is fixed within the surgicalenvironment 601. Alternatively, the insertion stage 728 can be movablewithin the surgical environment 601 but have a known location (e.g., viaa tracking sensor (not shown) or other tracking device) within thesurgical environment 601. In these alternatives, the medical instrumentframe of reference (XM, YM, ZM) is fixed or otherwise known relative tothe surgical frame of reference (XS, YS, ZS).

The medical instrument system 604 of FIG. 7 includes an elongate device731 (e.g., corresponding to elongated device 131 in FIG. 1 ), a medicalinstrument 732, an instrument body 735, at least a portion of thepositional sensor system 608, and at least a portion of the endoscopicimaging system 609. In some embodiments, the elongate device 731 is aflexible catheter or other biomedical device that defines a channel orlumen 744. The channel 744 can be sized and shaped to receive themedical instrument 732 (e.g., via a proximal end 736 of the elongatedevice 731 and/or an instrument port (not shown)) and facilitatedelivery of the medical instrument 732 to a distal portion 738 of theelongate device 731. The elongate device 731 is coupled to theinstrument body 735, which in turn is coupled and fixed relative to theinstrument carriage 726 of the manipulator assembly 602.

In operation, the manipulator assembly 602 can control insertion motion(e.g., proximal and/or distal motion along an axis A) of the elongatedevice 731 into the patient 603 via a natural or surgically createdanatomic orifice of the patient 603 to facilitate navigation of theelongate device 731 through anatomic passageways of an anatomic regionof the patient 603 and/or to facilitate delivery of a distal portion 738of the elongate device 731 to or near a target location within thepatient 603. For example, the instrument carriage 726 and/or theinsertion stage 728 may include actuators (not shown), such asservomotors, that facilitate control over motion of the instrumentcarriage 726 along the insertion stage 728. Additionally, oralternatively, the manipulator assembly 602 in some embodiments cancontrol motion of the distal portion 738 of the elongate device 731 inmultiple directions, including yaw, pitch, and roll rotationaldirections (e.g., to navigate patient anatomy). To this end, theelongate device 731 may house or include cables, linkages, and/or othersteering controls (not shown) that the manipulator assembly 602 can useto controllably bend the distal portion 738 of the elongate device 731.For example, the elongate device 731 can house at least four cables thatcan be used by the manipulator assembly 602 to provide (i) independent“up-down” steering to control a pitch of the distal portion 738 of theelongate device 731 and (ii) independent “left-right” steering of theelongate device 731 to control a yaw of the distal portion 738 of theelongate device 731.

The medical instrument 732 of the medical instrument system 604 can beused for medical procedures, such as for survey of anatomic passageways,surgery, biopsy, ablation, illumination, irrigation, and/or suction.Thus, the medical instrument 732 can include image capture probes,biopsy instruments, laser ablation fibers, and/or other surgical,diagnostic, and/or therapeutic tools. For example, the medicalinstrument 732 can include an endoscope or other biomedical devicehaving the one or more image capture devices 747 positioned at a distalportion 737 of and/or at other locations along the medical instrument732. In these embodiments, an image capture device 747 can capture oneor more real navigational images or video (e.g., a sequence of one ormore real navigational image frames) of anatomic passageways and/orother real patient anatomy while the medical instrument 732 is within ananatomic region of the patient 603.

As discussed above, the medical instrument 732 can be deployed intoand/or be delivered to a target location within the patient 603 via thechannel 744 defined by the elongate device 731. In embodiments in whichthe medical instrument 732 includes an endoscope or other biomedicaldevice having an image capture device 747 at its distal portion 737, theimage capture device 747 can be advanced to the distal portion 738 ofthe elongate device 731 before, during, and/or after the manipulatorassembly 602 navigates the distal portion 738 of the elongate device 731to a target location within the patient 603. In these embodiments, themedical instrument 732 can be used as a survey instrument to capturereal navigational images of anatomic passageways and/or other realpatient anatomy, and/or to aid an operator (not shown) to navigate thedistal portion 738 of the elongate device 731 through anatomicpassageways to the target location.

As another example, after the manipulator assembly 602 positions thedistal portion 738 of the elongate device 731 proximate a targetlocation within the patient 603, the medical instrument 732 can beadvanced beyond the distal portion 738 of the elongate device 731 toperform a medical procedure at the target location. Continuing with thisexample, after all or a portion of the medical procedure at the targetlocation is complete, the medical instrument 732 can be retracted backinto the elongate device 731 and, additionally or alternatively, beremoved from the proximal end 736 of the elongate device 731 or fromanother instrument port (not shown) along the elongate device 731.

As shown in FIG. 7 , the positional sensor system 608 of the medicalinstrument system 604 includes a shape sensor 733 and a positionmeasuring device 739. In these and other embodiments, the positionalsensor system 608 can include other position sensors (e.g.,accelerometers, rotary encoders, etc.) in addition to or in lieu of theshape sensor 733 and/or the position measuring device 739.

The shape sensor 733 of the positional sensor system 608 includes anoptical fiber extending within and aligned with the elongate device 731.In one embodiment, the optical fiber of the shape sensor 733 has adiameter of approximately 200 μm. In other embodiments, the diameter ofthe optical fiber may be larger or smaller. The optical fiber of theshape sensor 733 forms a fiber optic bend sensor that is used todetermine a shape, orientation, and/or pose of the elongate device 731.In some embodiments, optical fibers having Fiber Bragg Gratings (FBGs)can be used to provide strain measurements in structures in one or moredimensions. Various systems and methods for monitoring the shape andrelative position of an optical fiber in three dimensions are describedin further detail in U.S. Patent Application Publication No.2006/0013523 (filed Jul. 13, 2005) (disclosing fiber optic position andshape sensing device and method relating thereto); U.S. Pat. No.7,781,724 (filed on Sep. 26, 2006) (disclosing fiber-optic position andshape sensing device and method relating thereto); U.S. Pat. No.7,772,541 (filed on Mar. 12, 2008) (disclosing fiber-optic positionand/or shape sensing based on Rayleigh scatter); and U.S. Pat. No.6,389,187 (filed on Jun. 17, 1998) (disclosing optical fiber bendsensors), which are all incorporated by reference herein in theirentireties. In these and other embodiments, sensors of the presenttechnology may employ other suitable strain sensing techniques, such asRayleigh scattering, Raman scattering, Brillouin scattering, andFluorescence scattering. In these and still other embodiments, the shapeof the elongate device 731 may be determined using other techniques. Forexample, a history of the pose of the distal portion 738 of the elongatedevice 731 can be used to reconstruct the shape of elongate device 731over an interval of time.

In some embodiments, the shape sensor 733 is fixed at a proximal point734 on the instrument body 735 of the medical instrument system 604. Inoperation, for example, the shape sensor 733 measures a shape in themedical instrument reference frame (XM, YM, ZM) from the proximal point734 to another point along the optical fiber, such as the distal portion738 of the elongate device 731. The proximal point 734 of the shapesensor 733 may be movable along with instrument body 735 but thelocation of proximal point 734 may be known (e.g., via a tracking sensor(not shown) or other tracking device).

The position measuring device 739 of the positional sensor system 608provides information about the position of the instrument body 735 as itmoves along the insertion axis A on the insertion stage 728 of themanipulator assembly 602. In some embodiments, the position measuringdevice 739 includes resolvers, encoders, potentiometers, and/or othersensors that determine the rotation and/or orientation of actuators (notshown) controlling the motion of the instrument carriage 726 of themanipulator assembly 602 and, consequently, the motion of the instrumentbody 735 of the medical instrument system 604.

C. EXAMPLES

Several aspects of the present technology are set forth in the followingexamples. Although several aspects of the present technology are setforth in examples directed to systems, computer-readable mediums, andmethods, any of these aspects of the present technology can similarly beset forth in examples directed to any of systems, computer-readablemediums, and methods in other embodiments.

-   -   1. A system for determining divergence of an anatomic region        from an anatomic model of the anatomic region, the system        comprising: a medical device comprising a sensor, wherein the        medical device is insertable within a patient; and a computing        device in communication with the medical device, the computing        device comprising a processor, and a memory coupled to the        processor and storing instructions that, when executed by the        processor, cause the system to perform operations comprising:        receiving sensor data acquired by the sensor of the medical        device while the medical device is inserted within an anatomic        region of the patient and after the medical device has been        registered to an anatomic model of the anatomic region, wherein        the anatomic model is based on previously-obtained image data of        the anatomic region and includes a virtual path extending        throughout the anatomic model to an anatomic structure of        interest, and wherein the sensor data indicates a location of at        least a portion of the medical device, comparing the sensor data        to a corresponding portion of the virtual path, based at least        in part on the comparing, producing a divergence classifier        indicative of a divergence of the anatomic region from the        anatomic model, and generating an alert when the divergence        classifier exceeds a predetermined threshold.    -   2. The system of example 1 wherein the generating the alert        includes displaying a graphical user interface on a display in        communication with the computing device, wherein the graphical        user interface includes one or both of graphical and text        information indicative of a determination of the divergence of        the anatomic region from the anatomic model.    -   3. The system of example 2 wherein the graphical user interface        includes a prompt to command the system to update a location of        the anatomic region within the anatomic model.    -   4. The system of any one of examples 1-3 wherein the operations        further comprise updating a virtual location of the anatomic        region with respect to the anatomic model while the medical        device is within the anatomic region.    -   5. The system of example 4 wherein the produced divergence        classifier is indicative of a divergence of the anatomic        structure of interest from the anatomic model, and wherein the        operations further comprise updating a virtual location of the        anatomic structure of interest while the medical device is        within the anatomic region.    -   6. The system of example 5 wherein the updating the virtual        location of the anatomic structure includes: calculating one or        more divergence vectors between the corresponding portion of the        virtual path and the at least a portion of the medical device,        selecting one or more of the divergence vectors associated with        one or more positional points located along the at least a        portion of the medical device that are nearest to the anatomic        structure, fitting a curve to the selected one or more        divergence vectors, and extrapolating the curve to estimate a        new target position of the anatomic structure in x,y,z        coordinate points.    -   7. The system of example 6 wherein the updating the virtual        location of the anatomic structure further includes applying a        weighting value to at least one of the one or more divergence        vectors.    -   8. The system of any one of examples 1-7 wherein comparing        includes:    -   determining an offset between the corresponding portion of the        virtual path and the medical device.    -   9. The system of example 8 wherein determining the offset        includes generating one or more divergence vectors pointing from        the corresponding portion of the virtual path to the medical        device, and wherein the one or more divergence vectors are point        matched.    -   10. The system of any one of examples 1-9 wherein the operations        further comprise analyzing the sensor data to determine a shape        of the at least a portion of the medical device.

11. The system of example 10, wherein comparing the sensor data to thecorresponding portion of the virtual path includes comparing the shapeof the at least a portion of the medical device with a portion of thevirtual path.

-   -   12. The system of example 11 wherein the anatomic region        includes an anatomic passageway, and wherein the determining the        offset includes measuring a magnitude and a direction of the        anatomic passageway deformation to determine a plurality of        deformation vectors, and comparing the shape of the at least a        portion of the medical device in real time to the virtual path        of the anatomic passageways predetermined from the        previously-obtained image data based on the deformation vectors.    -   13. The system of any one of examples 1-12 wherein the        divergence classifier includes a device-to-path distance        parameter that includes a mean distance between a last region of        the portion of the virtual path to a distal end portion of the        medical device.    -   14. The system of any one of examples 1-13 wherein the sensor        data acquired by the sensor of the medical device are associated        with one or more of a position, orientation, speed, pose, and/or        shape of the medical device.    -   15. The system of any one of examples 1-14 wherein the medical        device includes a catheter, and wherein the sensor includes a        shape sensor comprising an optical fiber extending within and        aligned with an elongate portion of the catheter.    -   16. The system of example 15 wherein a plurality of points        associated with the shape of the medical device are determined        from sampled points by the shape sensor.    -   17. The system of any one of examples 1-16 wherein the medical        device includes a catheter, and wherein the sensor includes an        electromagnetic (EM) sensor located at a distal end or tip of        the catheter.    -   18. The system of example 17 wherein the plurality of points        associated with the shape of the medical device are determined        from a plurality of individual points measured at the distal end        or tip of the catheter by the EM sensor as it is driven through        an anatomic passageway.    -   19. The system of any one of examples 1-18 wherein the sensor of        the medical device is configured to generate one or both of        position sensor data and motion sensor data during data sampling        of the anatomic region of the patient, and wherein the        operations further comprise:    -   identifying an anatomical landmark of the anatomic region while        the medical device is navigating within the anatomic region of        the patient, implementing the comparing step to compare the        sensor data to the corresponding portion of the virtual path        associated with the anatomical landmark, and updating        registration of the medical device based, at least in part, on        the compared sensor data to the anatomical landmark.    -   20. A non-transitory, computer-readable medium storing        instructions thereon that, when executed by one or more        processors of a computing system, cause the computing system to        perform operations comprising:    -   receiving sensor data acquired by a sensor of a medical device        while the medical device is inserted within an anatomic region        of a patient and after the medical device has been registered to        an anatomic model of the anatomic region, wherein the anatomic        model is based on previously-obtained image data of the anatomic        region and includes a virtual path extending throughout the        anatomic model to an anatomic structure of interest, and wherein        the sensor data indicates a location of at least a portion of        the medical device, comparing the sensor data to a corresponding        portion of the virtual path, based at least in part on the        comparing, producing a divergence classifier indicative of a        divergence of the anatomic region from the anatomic model, and

D. CONCLUSION

The systems and methods described herein can be provided in the form oftangible and non-transitory machine-readable medium or media (such as ahard disk drive, hardware memory, etc.) having instructions recordedthereon for execution by a processor or computer. The set ofinstructions can include various commands that instruct the computer orprocessor to perform specific operations such as the methods andprocesses of the various embodiments described here. The set ofinstructions can be in the form of a software program or application.The computer storage media can include volatile and non-volatile media,and removable and non-removable media, for storage of information suchas computer-readable instructions, data structures, program modules orother data. The computer storage media can include, but are not limitedto, RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memorytechnology, CD-ROM, DVD, or other optical storage, magnetic diskstorage, or any other hardware medium which can be used to store desiredinformation and that can be accessed by components of the system.Components of the system can communicate with each other via wired orwireless communication. The components can be separate from each other,or various combinations of components can be integrated together into amonitor or processor or contained within a workstation with standardcomputer hardware (for example, processors, circuitry, logic circuits,memory, and the like). The system can include processing devices such asmicroprocessors, microcontrollers, integrated circuits, control units,storage media, and other hardware.

Although many of the embodiments are described above in the context ofnavigating and performing medical procedures within lungs of a patient,other applications and other embodiments in addition to those describedherein are within the scope of the present technology. For example,unless otherwise specified or made clear from context, the devices,systems, methods, and computer program products of the presenttechnology can be used for various image-guided medical procedures, suchas medical procedures performed on, in, or adjacent hollow patientanatomy, and, more specifically, in procedures for surveying, biopsying,ablating, or otherwise treating tissue within and/or proximal the hollowpatient anatomy. Thus, for example, the systems, devices, methods, andcomputer program products of the present disclosure can be used in oneor more medical procedures associated with other patient anatomy, suchas the bladder, urinary tract, GI system, and/or heart of a patient.

As used herein, the term “operator” shall be understood to include anytype of personnel who may be performing or assisting a medical procedureand, thus, is inclusive of a physician, a surgeon, a doctor, a nurse, amedical technician, other personnel or user of the technology disclosedherein, and any combination thereof. Additionally, or alternatively, theterm “patient” should be considered to include human and/or non-human(e.g., animal) patients upon which a medical procedure is beingperformed.

From the foregoing, it will be appreciated that specific embodiments ofthe technology have been described herein for purposes of illustration,but well-known structures and functions have not been shown or describedin detail to avoid unnecessarily obscuring the description of theembodiments of the technology. To the extent any materials incorporatedherein by reference conflict with the present disclosure, the presentdisclosure controls. Where the context permits, singular or plural termscan also include the plural or singular term, respectively. Moreover,unless the word “or” is expressly limited to mean only a single itemexclusive from the other items in reference to a list of two or moreitems, then the use of “or” in such a list is to be interpreted asincluding (a) any single item in the list, (b) all of the items in thelist, or (c) any combination of the items in the list. As used herein,the phrase “and/or” as in “A and/or B” refers to A alone, B alone, andboth A and B. Where the context permits, singular or plural terms canalso include the plural or singular term, respectively. Additionally,the terms “comprising,” “including,” “having” and “with” are usedthroughout to mean including at least the recited feature(s) such thatany greater number of the same feature and/or additional types of otherfeatures are not precluded.

Furthermore, as used herein, the term “substantially” refers to thecomplete or nearly complete extent or degree of an action,characteristic, property, state, structure, item, or result. Forexample, an object that is “substantially” enclosed would mean that theobject is either completely enclosed or nearly completely enclosed. Theexact allowable degree of deviation from absolute completeness may insome cases depend on the specific context. However, generally speakingthe nearness of completion will be so as to have the same overall resultas if absolute and total completion were obtained. The use of“substantially” is equally applicable when used in a negativeconnotation to refer to the complete or near complete lack of an action,characteristic, property, state, structure, item, or result.

The above detailed descriptions of embodiments of the technology are notintended to be exhaustive or to limit the technology to the precise formdisclosed above. Although specific embodiments of, and examples for, thetechnology are described above for illustrative purposes, variousequivalent modifications are possible within the scope of thetechnology, as those skilled in the relevant art will recognize. Forexample, while steps are presented in a given order, alternativeembodiments can perform steps in a different order. As another example,various components of the technology can be further divided intosubcomponents, and/or various components and/or functions of thetechnology can be combined and/or integrated. Furthermore, althoughadvantages associated with certain embodiments of the technology havebeen described in the context of those embodiments, other embodimentscan also exhibit such advantages, and not all embodiments neednecessarily exhibit such advantages to fall within the scope of thetechnology.

It should also be noted that other embodiments in addition to thosedisclosed herein are within the scope of the present technology. Forexample, embodiments of the present technology can have differentconfigurations, components, and/or procedures in addition to those shownor described herein. Moreover, a person of ordinary skill in the artwill understand that these and other embodiments can be without severalof the configurations, components, and/or procedures shown or describedherein without deviating from the present technology. Accordingly, thedisclosure and associated technology can encompass other embodiments notexpressly shown or described herein.

1. A system for determining divergence of an anatomic region from ananatomic model of the anatomic region, the system comprising: a medicaldevice comprising a sensor, wherein the medical device is insertablewithin a patient; and a computing device in communication with themedical device, the computing device comprising a processor, and amemory coupled to the processor and storing instructions that, whenexecuted by the processor, cause the system to perform operationscomprising: receiving sensor data acquired by the sensor of the medicaldevice while the medical device is inserted within an anatomic region ofthe patient and after the medical device has been registered to ananatomic model of the anatomic region, wherein the anatomic model isbased on previously-obtained image data of the anatomic region andincludes a virtual path extending throughout the anatomic model to ananatomic structure of interest, and wherein the sensor data indicates alocation of at least a portion of the medical device, comparing thesensor data to a corresponding portion of the virtual path, based atleast in part on the comparing, producing a divergence classifierindicative of a divergence of the anatomic region from the anatomicmodel, and generating an alert when the divergence classifier exceeds apredetermined threshold.
 2. The system of claim 1 wherein the generatingthe alert includes displaying a graphical user interface on a display incommunication with the computing device, wherein the graphical userinterface includes one or both of graphical and text informationindicative of a determination of the divergence of the anatomic regionfrom the anatomic model.
 3. The system of claim 2 wherein the graphicaluser interface includes a prompt to command the system to update alocation of the anatomic region within the anatomic model.
 4. The systemof claim 1 wherein the operations further comprise updating a virtuallocation of the anatomic region with respect to the anatomic model whilethe medical device is within the anatomic region.
 5. The system of claim4 wherein the produced divergence classifier is indicative of adivergence of the anatomic structure of interest from the anatomicmodel, and wherein the operations further comprise updating a virtuallocation of the anatomic structure of interest while the medical deviceis within the anatomic region.
 6. The system of claim 5 wherein theupdating the virtual location of the anatomic structure includes:calculating one or more divergence vectors between the correspondingportion of the virtual path and the at least a portion of the medicaldevice, selecting one or more of the divergence vectors associated withone or more positional points located along the at least a portion ofthe medical device that are nearest to the anatomic structure, fitting acurve to the selected one or more divergence vectors, and extrapolatingthe curve to estimate a new target position of the anatomic structure inx,y,z coordinate points.
 7. The system of claim 6 wherein the updatingthe virtual location of the anatomic structure further includes applyinga weighting value to at least one of the one or more divergence vectors.8. The system of claim 1 wherein comparing includes: determining anoffset between the corresponding portion of the virtual path and themedical device.
 9. The system of claim 8 wherein determining the offsetincludes generating one or more divergence vectors pointing from thecorresponding portion of the virtual path to the medical device, andwherein the one or more divergence vectors are point matched.
 10. Thesystem of claim 1 wherein the operations further comprise analyzing thesensor data to determine a shape of the at least a portion of themedical device.
 11. The system of claim 10, wherein comparing the sensordata to the corresponding portion of the virtual path includes comparingthe shape of the at least a portion of the medical device with a portionof the virtual path.
 12. The system of claim 11 wherein the anatomicregion includes an anatomic passageway, and wherein the determining theoffset includes measuring a magnitude and a direction of the anatomicpassageway deformation to determine a plurality of deformation vectors,and comparing the shape of the at least a portion of the medical devicein real time to the virtual path of the anatomic passagewayspredetermined from the previously-obtained image data based on thedeformation vectors.
 13. The system of claim 1 wherein the divergenceclassifier includes a device-to-path distance parameter that includes amean distance between a last region of the portion of the virtual pathto a distal end portion of the medical device.
 14. The system of claim 1wherein the sensor data acquired by the sensor of the medical device areassociated with one or more of a position, orientation, speed, pose,and/or shape of the medical device.
 15. The system of claim 1 whereinthe medical device includes a catheter, and wherein the sensor includesa shape sensor comprising an optical fiber extending within and alignedwith an elongate portion of the catheter.
 16. The system of claim 15wherein a plurality of points associated with the shape of the medicaldevice are determined from sampled points by the shape sensor.
 17. Thesystem of claim 1 wherein the medical device includes a catheter, andwherein the sensor includes an electromagnetic (EM) sensor located at adistal end or tip of the catheter.
 18. The system of claim 17 whereinthe plurality of points associated with the shape of the medical deviceare determined from a plurality of individual points measured at thedistal end or tip of the catheter by the EM sensor as it is driventhrough an anatomic passageway.
 19. The system of claim 1 wherein thesensor of the medical device is configured to generate one or both ofposition sensor data and motion sensor data during data sampling of theanatomic region of the patient, and wherein the operations furthercomprise: identifying an anatomical landmark of the anatomic regionwhile the medical device is navigating within the anatomic region of thepatient, implementing the comparing step to compare the sensor data tothe corresponding portion of the virtual path associated with theanatomical landmark, and updating registration of the medical devicebased, at least in part, on the compared sensor data to the anatomicallandmark.
 20. A non-transitory, computer-readable medium storinginstructions thereon that, when executed by one or more processors of acomputing system, cause the computing system to perform operationscomprising: receiving sensor data acquired by a sensor of a medicaldevice while the medical device is inserted within an anatomic region ofa patient and after the medical device has been registered to ananatomic model of the anatomic region, wherein the anatomic model isbased on previously-obtained image data of the anatomic region andincludes a virtual path extending throughout the anatomic model to ananatomic structure of interest, and wherein the sensor data indicates alocation of at least a portion of the medical device, comparing thesensor data to a corresponding portion of the virtual path, based atleast in part on the comparing, producing a divergence classifierindicative of a divergence of the anatomic region from the anatomicmodel, and generating an alert when the divergence classifier exceeds apredetermined threshold.